Ja n 20 10 DISTILLED SENSING : ADAPTIVE SAMPLING FOR SPARSE DETECTION AND ESTIMATION

Adaptive sampling results in dramatic improvements in the recovery of sparse signals in white Gaussian noise. A sequential adap-tive sampling-and-refinement procedure called distilled sensing (DS) is proposed and analyzed. DS is a form of multi-stage experimental design and testing. Because of the adaptive nature of the data collection, DS can detect and localize far weaker signals than possible from non-adaptive measurements. In particular, reliable detection and localization (support estimation) using non-adaptive samples is possible only if the signal amplitudes grow logarithmically with the problem dimension. Here it is shown that using adaptive sampling, reliable detection is possible provided the amplitude exceeds a constant , and localization is possible when the amplitude exceeds any arbitrarily slowly growing function of the dimension. 1. Introduction. In high dimensional multiple hypothesis testing problems the aim is to identify the subset of the hypotheses that differ from the null distribution, or simply to decide if one or more of the hypotheses do not follow the null. There is now a well developed theory and methodology for this problem, and the fundamental limitations in the high dimensional setting are quite clear. However, most existing treatments of the problem assume a non-adaptive measurement process. The question of how the limitations might differ under a more flexible, sequential adaptive measurement process has not been addressed. This paper shows that this additional flexibility can yield surprising and dramatic performance gains.